Managing ETL Processes

ETL tools allow the definition of sometimes complex processes to extract, transform, and load heterogeneous data into a data warehouse or to perform other data migration tasks. In larger organizations many ETL processes of different data integration projects are accumulated. Such processes can encompass common sub-processes, shared data sources and targets, and same or similar operations. However, there is no common method or approach to systematically manage such ETL processes. We propose the highlevel management of such processes as a generic approach to enable their flexible re-use, optimization, and rapid development. To this end we introduce a set of basic operators on ETL processes, such as merge or invert, and motivate their use in several scenarios.